A multimodal approach to coding discourse: Collaboration, distributed cognition, and geometric reasoning

Our research aims to identify children’s communicative strategies when faced with the task of solving a geometric puzzle in CSCL contexts. We investigated how to identify and trace distributed cognition in problem-solving interactions based on discursive cohesion to objects, participants, and prior discursive content, and geometric and cooperative concepts. We report on the development of a method of coding and representation of verbal and gestural content for multimodal interactional data and initial application of this framework to a microethnographic case study of two small groups of 7 and 8-year-old learners solving tangram manipulatives in physical and virtual desktop settings. We characterize the establishment of shared reference points as “coreferences” which cohere on object, para, and meta-levels through both gesture and speech. Our analysis foregrounds how participants establish common referential ground to facilitate collaborative problem solving with either computer-supported or physical puzzles. Using multimodal analysis and a theoretical framework we developed to study interactional dynamics, we identified patterns of focus, dominance, and coalition formation as they relate to coreferentiality on multiple levels. Initial findings indicate increased communication and cohesion to higher-level principles in the virtual tangram puzzle-solving setting. This work contributes to available models of multimodal analysis of distributed cognition using current manipulative technologies for early childhood mathematics education.